Informing pandemic response in the face of uncertainty. An evaluation of the U.S. COVID-19 Scenario Modeling Hub (preprint)

Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.

Challenges of COVID-19 Case Forecasting in the US, 2020-2021 (preprint)

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naive baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making. Author SummaryAs SARS-CoV-2 began to spread throughout the world in early 2020, modelers played a critical role in predicting how the epidemic could take shape. Short-term forecasts of epidemic outcomes (for example, infections, cases, hospitalizations, or deaths) provided useful information to support pandemic planning, resource allocation, and intervention. Yet, infectious disease forecasting is still a nascent science, and the reliability of different types of forecasts is unclear. We retrospectively evaluated COVID-19 case forecasts, which were often unreliable. For example, forecasts did not anticipate the speed of increase in cases in early winter 2020. This analysis provides insights on specific problems that could be addressed in future research to improve forecasts and their use. Identifying the strengths and weaknesses of forecasts is critical to improving forecasting for current and future public health responses.

Spatiotemporal Variations of "Triple-demic" Outbreaks of Respiratory Infections in the United States in the Post-COVID-19 Era (preprint)

Objectives: The United States confronted a "triple-demic" of influenza, respiratory syncytial virus, and COVID-19 in the winter of 2022, resulting in increased respiratory infections and a higher demand for medical supplies. It is urgent to analyze each epidemic and their co-occurrence in space and time to identify hotspots and provide insights for public health strategy. Methods: We used retrospective space-time scan statistics to retrospect the situation of COVID-19, influenza, and RSV in 51 US states from October 2021 to February 2022, and then applied prospective space-time scan statistics to monitor spatiotemporal variations of each individual epidemic, respectively and collectively from October 2022 to February 2023. Results: Our analysis indicated that compared to the winter of 2021, COVID-19 cases decreased while influenza and RSV infections increased significantly during the winter of 2022. We revealed that a twin-demic high-risk cluster of influenza and COVID-19 but no triple-demic clusters emerged during the winter of 2021. We further identified a large high-risk cluster of triple-demic in the central US from late November, with COVID-19, influenza, and RSV having relative risks of 1.14, 1.90, and 1.59, respectively. The number of states at high risk for multiple-demic increased from 15 in October 2022 to 21 in January 2023. Conclusion: Our study provides a novel spatiotemporal perspective to explore and monitor the transmission patterns of the triple epidemic, which could inform public health authorities' resource allocation to mitigate future outbreaks.

Characterizing human collective behaviours of COVID-19 in Hong Kong (preprint)

People are likely to engage in collective behaviour online during extreme events, such as the COVID-19 crisis, to express their awareness, actions and concerns. Hong Kong has implemented stringent public health and social measures (PHSMs) to curb COVID-19 epidemic waves since the first COVID-19 case was confirmed on 22 January 2020. People are likely to engage in collective behaviour online during extreme events, such as the COVID-19 crisis, to express their awareness, actions and concerns. Here, we offer a framework to evaluate interactions among individuals emotions, perception, and online behaviours in Hong Kong during the first two waves (February to June 2020) and found a strong correlation between online behaviours of Google search and the real-time reproduction numbers. To validate the model output of risk perception, we conducted 10 rounds of cross-sectional telephone surveys from February 1 through June 20 in 2020 to quantify risk perception levels over time. Compared with the survey results, the estimates of the risk perception of individuals using our network-based mechanistic model capture 80% of the trend of people risk perception (individuals who worried about being infected) during the studied period. We may need to reinvigorate the public by engaging people as part of the solution to live their lives with reduced risk.

The impact of COVID-19 vaccination in the US: averted burden of SARS-COV-2-related cases, hospitalizations and deaths (preprint)

By August 1, 2022, the SARS-CoV-2 virus had caused over 90 million cases of COVID-19 and one million deaths in the United States. Since December 2020, SARS-CoV-2 vaccines have been a key component of US pandemic response; however, the impacts of vaccination are not easily quantified. Here, we use a dynamic county-scale metapopulation model to estimate the number of cases, hospitalizations, and deaths averted due to vaccination during the first six months of vaccine availability. We estimate that COVID-19 vaccination was associated with over 8 million fewer confirmed cases, over 120 thousand fewer deaths, and 700 thousand fewer hospitalizations during the first six months of the campaign.

Impact of public health and social measures on hand-foot-mouth disease transmission and prediction of upcoming season after relaxation of COVID-19 control measures (preprint)

Background School based-measures such as school closure and school holidays have been considered a viable intervention during the hand-foot-mouth disease (HFMD) epidemic. However, the impact of public health and social measures (PHSMs) on HFMD transmission and the consequences of the HFMD epidemic after relaxing the PHSMs remains unclear.Methods Korean national surveillance data on HFMD from 2014 to 2019 was used to estimate the temporal changes in HFMD transmissibility (instantaneous reproductive number, \({R}_{t}\)). Furthermore, we used a simulation based stochastic transmission model to predict the future burden of HFMDs after relaxation of COVID-19 PHSMs, assuming all the Korean population was susceptible to HFMD in 2023, the stochastic transmission model simulated the number of HFMD cases in South Korea.Results We found that school holidays and 2015 PHSMs reduced the \({R}_{t}\) by 2–7% and 13%, respectively. Model projections indicated that the peak magnitude for season 2023, after relaxation of COVID-19 PHSMs in late 2022, could be 2·5–8·5 fold larger than that of the HFMD seasons during the pre-pandemic.Conclusions PHSMs likely have a larger impact on HFMD transmission than school-based measures alone (i.e. school holidays). Furthermore, complete relaxation of COVID-19 PHSMs could significantly increase the HFMD infection burden for the upcoming HFMD season. Preventive measures targeting preschoolers could be considered as potential options for reducing the future burden of HFMD.

Contact tracing reveals community transmission of COVID-19 in New York City (preprint)

Understanding SARS-CoV-2 transmission within and among communities is critical for tailoring public health policies to local context. However, analysis of community transmission is challenging due to a lack of high-resolution surveillance and testing data. Here, using contact tracing records for 644,029 cases and their contacts in New York City during the second pandemic wave, we provide a detailed characterization of the operational performance of contact tracing and reconstruct exposure and transmission networks at individual and ZIP code scales. We find considerable heterogeneity in reported close contacts and secondary infections and evidence of extensive transmission across ZIP code areas. Our analysis reveals the spatial pattern of SARS-CoV-2 spread and communities that are tightly interconnected by exposure and transmission. We find that higher vaccination coverage and reduced numbers of visitors to points-of-interest are associated with fewer within- and cross-ZIP code transmission events, highlighting potential measures for curtailing SARS-CoV-2 spread in urban settings.

Impact of SARS-CoV-2 vaccination of children ages 5-11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021-March 2022: a multi-model study (preprint)

Background: SARS-CoV-2 vaccination of persons aged 12 years and older has reduced disease burden in the United States. The COVID-19 Scenario Modeling Hub convened multiple modeling teams in September 2021 to project the impact of expanding vaccine administration to children 5-11 years old on anticipated COVID-19 burden and resilience against variant strains. Methods: Nine modeling teams contributed state- and national-level projections for weekly counts of cases, hospitalizations, and deaths in the United States for the period September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of: 1) presence vs. absence of vaccination of children ages 5-11 years starting on November 1, 2021; and 2) continued dominance of the Delta variant vs. emergence of a hypothetical more transmissible variant on November 15, 2021. Individual team projections were combined using linear pooling. The effect of childhood vaccination on overall and age-specific outcomes was estimated by meta-analysis approaches. Findings: Absent a new variant, COVID-19 cases, hospitalizations, and deaths among all ages were projected to decrease nationally through mid-March 2022. Under a set of specific assumptions, models projected that vaccination of children 5-11 years old was associated with reductions in all-age cumulative cases (7.2%, mean incidence ratio [IR] 0.928, 95% confidence interval [CI] 0.880-0.977), hospitalizations (8.7%, mean IR 0.913, 95% CI 0.834-0.992), and deaths (9.2%, mean IR 0.908, 95% CI 0.797-1.020) compared with scenarios where children were not vaccinated. This projected effect of vaccinating children 5-11 years old increased in the presence of a more transmissible variant, assuming no change in vaccine effectiveness by variant. Larger relative reductions in cumulative cases, hospitalizations, and deaths were observed for children than for the entire U.S. population. Substantial state-level variation was projected in epidemic trajectories, vaccine benefits, and variant impacts. Conclusions: Results from this multi-model aggregation study suggest that, under a specific set of scenario assumptions, expanding vaccination to children 5-11 years old would provide measurable direct benefits to this age group and indirect benefits to the all-age U.S. population, including resilience to more transmissible variants.

Evaluating the impact of quarantine measures on COVID-19 spread (preprint)

During the early stage of the COVID-19 pandemic, many countries implemented non-pharmaceutical interventions (NPIs) to control the transmission of SARS-CoV-2, the causative pathogen of COVID-19. Among those NPIs, quarantine measures were widely adopted and enforced through stay-at-home and shelter-in-place orders. Understanding the effectiveness of quarantine measures can inform decision-making and control planning during the ongoing COVID-19 pandemic and for future disease outbreaks. In this study, we use mathematical models to evaluate the impact of quarantine measures on COVID-19 spread in four cities that experienced large-scale outbreaks in the spring of 2020: Wuhan, New York, Milan, and London. We develop a susceptible-exposed-infected-removed (SEIR)-type model with a component of quarantine and couple this disease transmission model with a data assimilation method. By calibrating the model to case data, we estimate key epidemiological parameters before lockdown in each city. We further examine the impact of quarantine rates on COVID-19 spread after lockdown using model simulations. Results indicate that quarantine of susceptible and exposed individuals and undetected infections is necessary to contain the outbreak; however, the quarantine rates for these populations can be reduced through faster isolation of confirmed cases. We generate counterfactual simulations to estimate effectiveness of quarantine measures. Without quarantine measures, the cumulative confirmed cases could be 73, 22, 43 and 93 times higher than reported numbers within 40 days after lockdown in Wuhan, New York, Milan, and London. Our findings underscore the essential role of quarantine during the early phase of the pandemic.

Non-pharmaceutical interventions and inoculation rate shape SARS-CoV-2 vaccination campaign success

Nearly 1 year into the coronavirus disease 2019 pandemic, the first severe acute respiratory syndrome coronavirus 2 vaccines received emergency use authorisation and vaccination campaigns began. A number of factors can reduce the averted burden of cases and deaths due to vaccination. Here, we use a dynamic model, parametrised with Bayesian inference methods, to assess the effects of non-pharmaceutical interventions (NPIs) (such as social distancing, mask mandates, school and workplace closure), and vaccine administration and uptake rates on infections and deaths averted in the United States. We show that scenarios depicting higher compliance with NPIs avert more than 60% of infections and 70% of deaths during the period of vaccine administration, and that increasing the vaccination rate from 5 to 11 million people per week could increase the averted burden by more than one-third. These findings underscore the importance of maintaining NPIs and increasing vaccine administration rates.

Epidemic Management and Control Through Risk-Dependent Individual Contact Interventions (preprint)

Testing, contact tracing, and isolation (TTI) is an epidemic management and control approach that is difficult to implement at scale because it relies on manual tracing of contacts. Exposure notification apps have been developed to digitally scale up TTI by harnessing contact data obtained from mobile devices; however, exposure notification apps provide users only with limited binary information when they have been directly exposed to a known infection source. Here we demonstrate a scalable improvement to TTI and exposure notification apps that uses data assimilation (DA) on a contact network. Network DA exploits diverse sources of health data together with the proximity data from mobile devices that exposure notification apps rely upon. It provides users with continuously assessed individual risks of exposure and infection, which can form the basis for targeting individual contact interventions. Simulations of the early COVID-19 epidemic in New York City prove the concepts. In the simulations, network DA identifies up to a factor 2 more infections than contact tracing when both harness the same contact data and diagnostic test data. This remains true even when only a relatively small fraction of the population uses network DA. When a sufficiently large fraction of the population ($\gtrsim 75\%$) uses network DA and complies with individual contact interventions, targeting contact interventions with network DA reduces deaths by up to a factor 4 relative to TTI. Network DA can be implemented by expanding the computational backend of existing exposure notification apps, thus greatly enhancing their capabilities. Implemented at scale, it has the potential to precisely and effectively control future epidemics while minimizing economic disruption.

Projected resurgence of COVID-19 in the United States in July-December 2021 resulting from the increased transmissibility of the Delta variant and faltering vaccination (preprint)

What is already known about this topic?The highly transmissible SARS-CoV-2 Delta variant has begun to cause increases in cases, hospitalizations, and deaths in parts of the United States. With slowed vaccination uptake, this novel variant is expected to increase the risk of pandemic resurgence in the US in July--December 2021. What is added by this report?Data from nine mechanistic models project substantial resurgences of COVID-19 across the US resulting from the more transmissible Delta variant. These resurgences, which have now been observed in most states, were projected to occur across most of the US, coinciding with school and business reopening. Reaching higher vaccine coverage in July--December 2021 reduces the size and duration of the projected resurgence substantially. The expected impact of the outbreak is largely concentrated in a subset of states with lower vaccination coverage. What are the implications for public health practice?Renewed efforts to increase vaccination uptake are critical to limiting transmission and disease, particularly in states with lower current vaccination coverage. Reaching higher vaccination goals in the coming months can potentially avert 1.5 million cases and 21,000 deaths and improve the ability to safely resume social contacts, and educational and business activities. Continued or renewed non-pharmaceutical interventions, including masking, can also help limit transmission, particularly as schools and businesses reopen.

Pandemic fatigue impedes mitigation of COVID-19 in Hong Kong (preprint)

Community-wide social distancing has been a cornerstone of pandemic control prior to mass vaccinations. The extent to which pandemic fatigue is undermining adherence to such measures and accelerating transmission remains unclear. Using large-scale weekly telephone surveys and mobility data, we characterize the evolution of risk perception and protective behaviours in Hong Kong. We estimate a 1.5% to 5.5% reduction in population compliance with protective policies for the fourth wave (October 2020 to January 2021) versus the third wave (July to August 2020), inducing prolonged disease circulation with increased infections. Mathematical models incorporating population protective behaviours estimates that the fourth wave would have been 14% smaller if not for pandemic fatigue. Mitigating pandemic fatigue is essential in maintaining population protective behaviours for controlling COVID-19.

Non-pharmaceutical interventions and inoculation rate shape SARS-COV-2 vaccination campaign success (preprint)

Nearly one year into the COVID-19 pandemic, the first SARS-COV-2 vaccines received emergency use authorization and vaccination campaigns began. A number of factors can reduce the averted burden of cases and deaths due to vaccination. Here, we use a dynamic model, parametrized with Bayesian inference methods, to assess the effects of non-pharmaceutical interventions, and vaccine administration and uptake rates on infections and deaths averted in the United States. We estimate that high compliance with non-pharmaceutical interventions could avert more than 60% of infections and 70% of deaths during the period of vaccine administration, and that increasing the vaccination rate from 5 to 11 million people per week could increase the averted burden by more than one third. These findings underscore the importance of maintaining non-pharmaceutical interventions and increasing vaccine administration rates.

Overall burden and characteristics of COVID-19 in the United States during 2020 (preprint)

The COVID-19 pandemic disrupted health systems and economies throughout the world during 2020 and was particularly devastating for the United States. Many of epidemiological features that produced observed rates of morbidity and mortality have not been thoroughly assessed. Here we use a data-driven model-inference approach to simulate the pandemic at county-scale in the United States during 2020 and estimate critical, time-varying epidemiological properties underpinning the dynamics of the virus. The pandemic in the US during 2020 was characterized by an overall ascertainment rate of 21.6% (95% credible interval (CI):18.9-25.5%). Population susceptibility at year end was 68.8% (63.4-75.3%), indicating roughly one third of the US population had been infected. Community infectious rates, the percentage of people harboring a contagious infection, rose above 0.8% (0.6-1.0%) before the end of the year, and were as high as 2.4% in some major metropolitan areas. In contrast, the infection fatality rate fell to 0.3% by year end; however, community control of transmission, estimated from trends of the time-varying reproduction number, Rt, slackened during successive pandemic waves. In the coming months, as vaccines are distributed and administered and new more transmissible virus variants emerge and spread, greater use of non-pharmaceutical interventions will be needed.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US (preprint)

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. In 2020, the COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized hundreds of thousands of specific predictions from more than 50 different academic, industry, and independent research groups. This manuscript systematically evaluates 23 models that regularly submitted forecasts of reported weekly incident COVID-19 mortality counts in the US at the state and national level. One of these models was a multi-model ensemble that combined all available forecasts each week. The performance of individual models showed high variability across time, geospatial units, and forecast horizons. Half of the models evaluated showed better accuracy than a naive baseline model. In combining the forecasts from all teams, the ensemble showed the best overall probabilistic accuracy of any model. Forecast accuracy degraded as models made predictions farther into the future, with probabilistic accuracy at a 20-week horizon more than 5 times worse than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. f

The importance of continued non-pharmaceutical interventions during the upcoming SARS-COV-2 vaccination campaign (preprint)

In this communication we assess the potential benefit of SARS-COV-2 pandemic vaccination in the US and show how continued use of non-pharmaceutical interventions (NPIs) will be crucial during implementation.

Role of air temperature and humidity in the transmission of SARS-CoV-2 in the United States (preprint)

Improved understanding of the effects of meteorological conditions on the transmission of SARS-CoV-2, the causative agent for COVID-19 disease, is urgently needed to inform mitigation efforts. Here, we estimated the relationship between air temperature or specific humidity (SH) and SARS-CoV-2 transmission in 913 U.S. counties with abundant reported infections from March 15 to August 31, 2020. Specifically, we quantified the associations of daily mean temperature and SH with daily estimates of the SARS-CoV-2 reproduction number (Rt) and calculated the fraction of Rt attributable to these meteorological conditions. Both lower temperature and lower SH were significantly associated with increased Rt. The fraction of Rt attributable to temperature was 5.10% (95% eCI: 5.00 - 5.18%), and the fraction of Rt attributable to SH was 14.47% (95% eCI: 14.37 - 14.54%). These fractions generally were higher in northern counties than in southern counties. Our findings indicate that cold and dry weather are moderately associated with increased SARS-CoV-2 transmissibility, with humidity playing a larger role than temperature.

Compound risks of hurricane evacuation amid the COVID-19 pandemic in the United States (preprint)

Current projections and unprecedented storm activity to date suggest the 2020 Atlantic hurricane season will be extremely active and that a major hurricane could make landfall during the global COVID-19 pandemic. Such an event would necessitate a large-scale evacuation, with implications for the trajectory of the pandemic. Here we model how a hypothetical hurricane evacuation from four counties in southeast Florida would affect COVID-19 case levels. We find that hurricane evacuation increases the total number of COVID-19 cases in both origin and destination locations; however, if transmission rates in destination counties can be kept from rising during evacuation, excess evacuation-induced case numbers can be minimized by directing evacuees to counties experiencing lower COVID-19 transmission rates. Ultimately, the number of excess COVID-19 cases produced by the evacuation depends on the ability of destination counties to meet evacuee needs while minimizing virus exposure through public health directives.

Differential Effects of Intervention Timing on COVID-19 Spread in the United States (preprint)

Assessing the effects of early non-pharmaceutical interventions1-5 on COVID-19 spread in the United States is crucial for understanding and planning future control measures to combat the ongoing pandemic6-10. Here we use county-level observations of reported infections and deaths11, in conjunction with human mobility data12 and a metapopulation transmission model13,14, to quantify changes of disease transmission rates in US counties from March 15, 2020 to May 3, 2020. We find significant reductions of the basic reproductive numbers in major metropolitan areas in association with social distancing and other control measures. Counterfactual simulations indicate that, had these same control measures been implemented just 1-2 weeks earlier, a substantial number of cases and deaths could have been averted. Specifically, nationwide, 56.5% [95% CI: 48.1%-65.9%] of reported infections and 54.0% [95% CI: 43.6%-63.8%] of reported deaths as of May 3, 2020 could have been avoided if the same control measures had been implemented just one week earlier. We also examine the effects of delays in re-implementing social distancing following a relaxation of control measures. A longer response time results in a stronger rebound of infections and death. Our findings underscore the importance of early intervention and aggressive response in controlling the COVID-19 pandemic.